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1. INTRODUCTION 

We are engaged in a project to produce a 30m impervious cover data set of the entire Earth for the years 
2000 and 2010 based on the Landsat Global Land Survey (GLS) data set. The GLS data from Landsat provide an 
unprecedented opportunity to map global urbanization at this resolution for the first time, with unprecedented 
detail and accuracy. Moreover, the spatial resolution of Landsat is absolutely essential to accurately resolve urban 
targets such as buildings, roads and parking lots. Finally, with GLS data available for the 1975, 1990, 2000, and 
2005 time periods, and soon for the 2010 period, the land cover/use changes due to urbanization can now be 
quantified at this spatial scale as well. 

Our approach works across spatial scales using very high spatial resolution commercial satellite data to both 
produce and evaluate continental scale products at the 30m spatial resolution of Landsat data. We are developing 
continental scale training data at lm or so resolution and aggregating these to 30m for training a regression tree 
algorithm. Because the quality of the input training data are critical, we have developed an interactive software 
tool, called HSegLeam, to facilitate the photo-interpretation of high resolution imagery data, such as Quickbird or 
Ikonos data, into an impervious versus non-impervious map. Previous work has shown that photo-interpretation 
of high resolution data at 1 meter resolution will generate an accurate 30m resolution ground reference when 
coarsened to that resolution. Since this process can be very time consuming when using standard clustering 
classification algorithms, we are looking at image segmentation as a potential avenue to not only improve the 
training process but also provide a semi-automated approach for generating the ground reference data. 

HSegLeam takes as its input a hierarchical set of image segmentations produced by the HSeg image 
segmentation program [1, 2]. HSegLeam lets an analyst specify pixel locations as being either positive or 
negative examples, and displays a classification of the study area based on these examples. For our study, the 
positive examples are examples of impervious surfaces and negative examples are examples of non-impervious 
surfaces. HSegLeam searches the hierarchical segmentation from HSeg for the coarsest level of segmentation at 
which selected positive example locations do not conflict with negative example locations and labels the image 
accordingly. The negative example regions are always defined at the finest level of segmentation detail. The 
resulting classification map can be then further edited at a region object level using the previously developed 
HSegViewer tool [3], 



After providing an overview of the HSeg image segmentation program, we provide a detailed description of 
the HSegLeam software tool. We then give examples of using HSegLeam to generate ground reference data and 
conclude with comments on the effectiveness of the HSegLeam tool. 

2. THE HSEG IMAGE SEGMENTATION PROGRAM 

A popular approach for perfomiing image segmentation is best merge region growing. An early example is 
the Hierarchical StepWise Optimization (HSWO) approach of Beaulieu and Goldberg [4]. HSWO is an iterative 
form of region growing, in which the iterations consist of finding the most optimal or best segmentation with one 
region less than the current segmentation. 

In complex scenes, such as remotely sensed images of the Earth, objects with similar spectral signatures 
(e.g., lakes, agricultural fields, buildings, etc.) appear in spatially separated locations. In such cases, it is useful to 
aggregate these spectrally similar but spatially disjoint region objects together into groups of regions objects that 
we call region classes. This aggregation may be performed as a post-processing step. However, best merge region 
growing, as exemplified by HSWO, may be modified to integrate this aggregation directly into the region 
growing process. This is the basis of the Hierarchical Segmentation (HSeg) algorithm. 

HSeg produces a segmentation hierarchy a set of several image segmentations at different levels of detail in 
which the segmentations at coarser levels of detail can be produced from simple merges of regions at finer levels 
of detail. This hierarchy may be useful for applications that require different levels of image segmentation details 
depending on the characteristics of the particular image objects segmented. An important feature of a 
segmentation hierarchy that distinguishes it from most other multilevel representations is that the segment or 
region boundaries are maintained at the full image spatial resolution for all levels of the segmentation hierarchy. 

The key unique aspect of HSeg is its tight intertwining of region growing segmentation, which produces 
spatially connected region objects, with non-adjacent region object aggregation, which groups sets of region 
objects together into region classes. HSegLeam takes advantage of this region object classification to extend its 
positive and negative example labeling throughout the entire image based on just a few examples. 

3, THE HSEGLEARN PHOTO-INTERPRETATION TOOL 

HSegLeam provides a convenient graphical user interface for designating region classes from the HSeg 
segmentation hierarchy as positive and negative example region classes. One way an analyst can do this is by 
selecting a pixel location in a display panel showing an RGB rendition of the image data or in a display panel of 
the current image labeling, and having HSegLeam highlight (in yellow) the region class containing the selected 
pixel. The analyst can either then select another region class for highlighting, or submit the currently highlighted 
region class as a positive or negative example class. Another way an analyst can highlight region classes is by 
selecting the “Circle Region of Interest” option on one of the display panels and circling an area in the display 
panel. HSegLeam will then highlight every region class contained within the circled area. The analyst can either 



then select another region class or group of region classes for highlighting, or submit the currently highlighted 
region classes as positive or negative example classes. 

Region classes submitted as negative examples are displayed in red at the finest resolution of the 
segmentation hierarchy. However, region classes submitted as positive examples are displayed in green at the 
coarsest resolution of the segmentation hierarchy at which the region class spatial extent does not overlap any 
submitted negative example region class at the finest resolution of the segmentation hierarchy. For the case for 
which a region class is submitted as both a positive and negative example, this region class is designated as 
ambiguous and colored purple in the display. An ambiguous class designation can be overridden by a subsequent 
resubmission as either a positive or negative example class. 

4. GENERATING GROUND REFERENCE DATA WITH HSEGLEARN 

In this section we provide an example of the generation of ground reference data with the HSegLeam photo- 
interpretation tool. Our test scene, displayed in Fig. 1(a), is an Ikonos image from downtown Baltimore, MD pan- 
sharpened to 1 -meter spatial resolution that was obtained from the Global Land Cover Facility (GLCF) at the 
University of Maryland. Using HSegLeam, we highlighted and selected several negative example region classes 
by circling four representative non-impervious areas throughout the scene (a section of the inner harbor waters, a 
section of the M & T Ba nk Stadium playing field, a wooded area in the far southwest comer, and a dry grassy 
area in the south, southwest portion of the image). Then we selected a pixel at the center of a large building in the 
north central portion of the image (the Baltimore Convention Center), highlighted the region class containing this 
pixel as a positive example region class, and submitted the highlighted negative and positive examples. The result 
is shown in Fig. 1(b). The result after submitting 20 more positive region class examples (such as other buildings, 
parking lots and roads) and a total of 44 negative region class examples is displayed in Fig. 1(c). One additional 
region class that was submitted as both a negative and positive example ended up as an ambiguous class, which is 
colored purple in Fig. 1(c). Individual ambiguous region objects may be relabeled as either positive or negative 
using HSegViewer on the output from HSegLeam. 

5. DISCUSSION 

The HSegLeam tool solves a key problem we have encountered with other approaches we have attempted to 
use for generating ground reference data from high spatial resolution imagery. This key problem was how to 
define the most effective level of segmentation or clustering detail. When too few clusters or region segments 
were selected, the classes of interest could not be separated, but when too many clusters of region segments were 
selected, the labeling of the classes of interest became very inefficient. HSegLeam automatically selects the most 
effective level of segmentation detail for labeling the classes of interest (for a binary positive class/negative class 
labeling problem) based on a simple selection of positive and negative examples by an analyst. 




(a) (b) (c) 

Fig. 1. (a) Ikonos test scene pan-sharpened to 1 meter spatial resolution from over Baltimore, MD. (b) HSegLeam 
positive (green) and negative (red) labeling after submitting several negative region class examples and one 
positive region class example from a building pixel location, (c) HSegLeam result after the submission of 21 
positive, 44 negative, and 1 ambiguous region class examples. Color key: Green for positive, red for negative and 
purple for ambiguous region classes. Black areas are unlabeled, but can be considered to be by default negative in 
the completed labeling shown in (c). 


The effectiveness HSegLeam’ s automatic selection of level of segmentation detail is demonstrated in the 
example discussed in the previous section. With the selection of just one positive example region class, which at 
the finest level of detail in the HSeg segmentation hierarchy just covered less than 25% of the roof of one large 
building, we obtained a much more generalized labeling of impervious surfaces including not just other spectrally 
similar roofs, but also roads and other manmade structures, as exhibited in Fig. 1(b). This appropriate level of 
segmentation detail was defined by the counter selection of just four areas of non-impervious surfaces. 

At 1GARSS 2012 we will provide a complete report on our development and testing of our HSegLeam tool 
with various commercial satellite data representing various levels of urbanization. 
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